Computers, computer operating systems, computer file systems, computer storage devices, and other computerized items may prefer to communicate using standard, fixed size blocks of data. For example, storage devices may require access to be in multiples of a pre-determined fixed block size and may only be addressable at block boundaries. File systems generally organize storage as a set of fixed size units. The file system may be helped by an operating system that may also prefer to write data using the pre-determined, fixed block size. Write efficiency in these systems can be negatively impacted when the data supplied by the application to be written does not completely fill the fixed sized block. The inefficiency arises because a file system or other actor may be forced to “fill in the gaps” in the write with additional data that may, unfortunately, be acquired by doing a read operation. Doing a read(s) as part of a write may be inefficient due to additional device latencies introduced.
Computer input/output (i/o) may be performed using buffered i/o. Computer i/o may also be performed using direct i/o. In buffered i/o, data to be written by a file system write may be buffered or cached in the operating system. Buffering the write provides opportunities to improve efficiencies by, for example, re-arranging write orders to achieve more sequential i/o to devices, re-ordering data to be written to reduce assembly issues on a receiving side, reducing the likelihood that blocks are repeatedly written, and so on. In direct i/o, data to be written by a file system write goes directly to a storage device and bypasses file system buffer caches in an operating system.
In a buffered output system, where there is a cache into which data to be written can be temporarily stored before being written, the read during write inefficiency may be addressed by aggregating the output of multiple writes in the cache. However, in a direct output environment, where there is no output cache in which padding or initialization approaches can be performed, it may be difficult, if even possible at all, for conventional systems to avoid the read during write occurrence and the resulting inefficiency in all situations.
FIG. 1 illustrates four situations including a block 100 that is completely filled with data, a block 110 having a gap 112 at its end, a block 120 having a gap 122 at its beginning, and a block 130 having a gap 132 at its beginning and a gap 134 at its end. In a conventional file system with conventional pre-write caching, gaps may be filled by the file system or by the operating system or by some other actor so that a full block is prepared and written. In one example, the padding may simply be null values (e.g., zeroes). In other examples, unfortunately, the padding may be other live data. Packaging two or more pieces of unrelated data into a single block may create new issues even while minimizing the read during write inefficiency. These packaging issues may complicate and/or compromise some processing (e.g., parity calculations) associated with redundant arrays of independent disks (RAID).
In a conventional direct i/o system, where there is no pre-write caching, the gaps may also be filled by a file system or by an operating system or by some other actor. Unfortunately, the gap filling may include performing a read operation to find data to fill the gap. Performing a read(s) during a write operation can significantly slow down the write operation. This may be particularly onerous during a read modify write cycle in RAIDs where the size of the block becomes very large in relation to the typical file i/o size.
Occasional write inefficiency may be more acceptable than lengthy, complicated, efforts to reduce inefficiency that may introduce more inefficiencies than they resolve. However, the combination of certain files and certain applications in certain environments may create conditions that cause the read during write inefficiency to occur at an unacceptable frequency. For example, some files (e.g., video files) have grown to large sizes and continue to grow even as very demanding applications (e.g., video capture) make ever increasing use of these large files in increasingly demanding environments (e.g., shared disk file system). Additionally, video capture can create issues. Video capture is now frequently performed using individual frame files rather than large media files that represent a whole movie or scene. The individual frame files have a data size governed by the video format. The individual frame files are rarely complete file system block multiples, and are practically never a complete RAID stripe multiple. By allocating files next to each other, and by filling in the portion of the allocated space that was not provided by the user, large amounts of sequential data can be written. This facilitates making the individual smaller files efficient for the RAID to perform parity calculations on.
Some files are input and output (e.g., read, written) in conventional known ways that produce a knowable or at least predictable context for a file. This context allows past behavior to predict future behavior, which facilitates addressing some i/o inefficiencies. However, large video files tend to be accessed in ways that do not produce predictable contexts. Thus, the combination of large file size and non-contextual usage may have frustrated conventional i/o optimization strategies. Even worse, some video files may be written in ways that frequently generate less than full block i/o. Therefore, the combination of large file size, non-contextual usage, and demanding applications that produce less than full block i/o may increase the frequency of read during write operations and consequently negatively impact i/o efficiency.
This situation may be further exacerbated due to direct i/o performed by specialized file systems. A conventional file system may use a cache and may perform a limited amount of i/o on a limited number of files over a period of time. However, a shared disk file system that facilitates sharing large files among many users who may all have different access times and/or approaches may face a more challenging task. This type of shared disk file system may deal with very large amounts of data (e.g., an entire file system backup, an entire file system replication, streaming a large number of movies to a large number of users) that could overwhelm a conventional cache based system. Since a conventional cache based system may be overwhelmed in this environment, direct i/o may be employed. However, as described above, direct i/o can be negatively impacted by read during write occurrences. The frequency of read during write occurrences may increase beyond a desired threshold when large, non-contextual video files are either present in or dominate the data being handled via direct i/o in a shared disk file system. The read during write issue may exist at different levels. At one level, the file system may need to issue a read to initialize parts of a block. At another level, a RAID system may need to issue an internal device read to complete a parity calculation because the file system did not supply a complete raid stripe worth of content. Example systems and methods address both levels.
When writing more than a single block of data, it may be desirable to do sequential i/o to reduce and/or minimize write head movement and thus to reduce and/or minimize write time. Performing a read during a sequential write could negatively impact many of the reasons for performing sequential i/o, with the damage becoming greater as the number of potential consecutive writes grows larger. Applications like replication, de-duplication, and tiered archival storage that run on shared disk file systems may generate a large volume of sequential i/o. However, these types of applications may also produce a large volume of efficiency destroying read during write occurrences. Similarly, video ingest applications may generate a large volume of sequential i/o as movies are scanned. Video ingest may produce a large volume of efficiency destroying read during write occurrences because, in some examples, video frames may not always match block sizes and will rarely, if ever, supply a complete RAID stripe worth of content.
Thus, there are significant read during write issues associated with actors that are performing writes. However, read during write occurrences may also occur at a storage device that receives less than a complete block of data. Consider RAID devices that may seek to perform operations like parity computations on fixed size pieces of data. If a RAID device is provided with less than a fixed size piece of data, then the RAID device may perform a read during its parity computation as part of its write to fill in its gaps. As described above, buffered i/o may not experience these read during write occurrences. Even if buffered i/o is available in a file system, there may still be read during write issues associated with RAID unless a file system can fill a whole RAID stripe before having to write the RAID's cache.